
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 1 3-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/723,169 


1 1/27/2000 


David H Sitrick 


STD 1684 


4193 



20787 



7590 



01/15/2004 

SITRICK & SITRICK 

8340 N LINCOLN AVENUE SUITE 201 

SKOKIE, IL 60077 



EXAMINER 



BECKER, SHAWN M 



ART UNIT 



PAPER NUMBER 



2173 

DATE MAILED: 01/15/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application No. 

09/723,169 



Examiner 

Shawn M. Becker 



Applicant(s) 

SITRICK, DAVID H 



Art Unit 

2173 



-- The MA/LING DATE of this communication appears on the cover sh et with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
• Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 

earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 04 November 2003 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-45 and 56-58 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-45 and 56-58 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) H The drawing(s) filed on 04 November 2003 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)DAII b)Q Some*c)D None of: 

Certified copies of the priority documents have been received. 
Certified copies of the priority documents have been received in Application No. . 



1.D 

3-D 



Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 
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DETAILED ACTION 
Drawings 

1. The drawings were received on 1 1/4/03. These drawings are accepted. 

Claim Rejections - 35 USC §112 



2. The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

3. Claims 20 and 58 are rejected under 35 U.S. C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

4. Claim 20 recites the limitation "the correlation means" in line 2 of the claim. There is 
insufficient antecedent basis for this limitation in the claim, 

5. Claim 58 recites the limitation "the frames" in lines 2-3 of the claim. There is 
insufficient antecedent basis for this limitation in the claim. 



Claim Rejections - 35 USC §102 



6. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
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subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

7. Claim 57 is rejected under 35 U.S.C. 102(e) as being anticipated by U.S. Patent No. 
6,504,546 to Cosatto et al. (hereinafter Cosatto). 

Referring to claim 57, Cosatto discloses a method of displaying an integrated display audiovisual 
presentation that provides an audiovisual presentation comprising at least a selected portion and 
having at least on reference point associated therewith, and having a defined orientation. See 
col. 5, lines 25-40. 

Cosatto provides at least two user images, each having a defined user image orientation 
and each user image having a pixel texture and a user object geometry. See col. 6, lines 7-20 and 
the section describing population of a database for samples of facial features, beginning on col. 
10, line 19. 

Cosatto identifies and selects one of the user images as a selected replacement for the 
selected portion responsive to analysis of the defined reference orientation and the defined user 
image orientations. See col. 5, lines 41-64, which describe how bitmap images from recorded 
video (user audiovisual information) are used to replace parts of the face (replacement parts) at 
an appropriate location on a face. Also see the description of Pose Estimation, beginning on col. 
9, line 20. 

Cosatto replaces the selected portion with the selected one of the user images responsive 
to the analysis, and to an association determined responsive to correlation of at least one selected 
reference point of the reference object with at least one selected replacement point of the one 
user image. See Fig. 7, which shows facial features integrated into a separate base face, 
replacing the previous facial features. Also, see col. 6, lines 7-20. 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. Claims 1-15, 17-45, 56, and 58 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent No. 6,504,546 to Cosatto et al. (hereinafter Cosatto) and U.S. Patent No. 
6,307,568 to Rom. 

Referring to claims 1, 26, and 37, Cosatto discloses a system and method for producing 
an integrated display audiovisual presentation with a source of first audiovisual presentation. 
See col. 3, lines 34-53, which describe how video is taken of a model along with an audio track. 
The first audiovisual presentation comprises at least one reference point and at least one 
reference object, wherein at least a selected one reference point is associated with a location on a 
selected one reference object having a placement and orientation. See col. 5, lines 25-55, which 
describe how facial parts are associated with their appropriate location on the face in accordance 
with an orientation. 

Cosatto discloses a source of user audiovisual information, wherein the user audiovisual 
information comprises pixel texture and user object geometry information for user objects 
representative of user images (see col. 6, lines 6-50, which describe using sample bitmaps of a 
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user's facial parts for texture mapping within the geometry of the face), the user object geometry 
information comprising at least one replacement point and at least one replacement object image, 
wherein at least a selected one replacement point is associated with a location on a selected 
replacement object image. See col. 5, lines 41-64, which describe how bitmap images from 
recorded video (user audiovisual information) are used replace parts of the face (replacement 
parts) at an appropriate location on a face. 

Cosatto discloses replacing part of the audiovisual presentation with an object from the 
user object audiovisual information based on the selected reference point (i.e. lips) and 
composting means responsive to the first audiovisual presentation, the pixel texture, and the user 
object information for reconstructing an appearance of the user object in the same placement as 
the selected one reference object to provide an integrated display audiovisual presentation, 
wherein at least a selected portion of the user image replaces a respective selected portion of the 
first audiovisual presentation. See Fig. 7, which shows facial features integrated into a separate 
base face, replacing the previous facial features. Also, see col. 6, lines 7-20. 

Although Cosatto discloses replacing a reference object with the best fit sample of user 
object information (i.e. col. 4, lines 15-22) and smoothing and blending the samples which hints 
to transforming the geometry of the user object information for a smooth transition into the first 
audiovisual presentation (col. 3, lines 34-53), Cosatto does not explicitly disclose processing 
means responsive to the first audiovisual presentation and the user audiovisual information, for 
geometrically transforming the user object geometry information to provide transformed user 
object information responsive to the selected one reference point. 
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However, Rom discloses a method of replacing reference objects (i.e. images of clothing 
originally on the body) in a presentation with object information (i.e. images of clothing to be 
"tried on"), and is thus similar to Cosatto image replacement and integration method. See Rom at 
col. 1, lines 45-63. Rom discloses using critical points similar to the reference points of Cosatto, 
wherein the critical points of Rom are used to adjust the spatial configuration of the clothing 
(object information) to fit the body of each user (reference object). The spatial configuration of 
Rom determines the geometry of the clothing (object information); therefore, adjusting the 
spatial configuration is a geometrical transform of object geometry information to fit the 
geometry and orientation of the reference object (i.e. body), wherein the object information is 
placed on the reference object and integrated (forms a composite) into the original audiovisual 
presentation. See Rom at col. 2, lines 20-33 and col. 4, lines 1-11. 

It would have been obvious to one of ordinary skill in the art with the teachings of 
Cosatto and Rom before him at the time of the invention to modify the image substitution and 
integration method of Cosatto to include geometrically transforming (i.e. adjusting spatial 
configuration of) the user object geometry information responsive to the reference object, as 
taught by Rom, in order to achieve a more accurate, realistic model in the animation, wherein the 
user information smoothly blends into the original presentation (i.e. base model). 

Referring to claims 9-10, the first audiovisual presentation of Cosatto comprises a time- 
ordered sequence of video images digitized and encoded into an encoded digital signal, that 
comprises a plurality of Motion Picture Expert's Group motion vectors, wherein the processing 
means is further responsive to at least a selected one of the plurality of MPEG motion vectors. 
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See col. 53-56, which discusses how the facial 
subsystem. 
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are handled by a model for the MPEG4 



Referring to claim 2, the first audiovisual presentation of Cosatto further comprises a 
video input signal. See col. 3, line 38 and col. 4, line 15. 

Referring to claim 3, the video input signal of Cosatto is a time-ordered sequence of 
video images. See col. 6, line 66 - col. 7, line 3. Also, the animations of the video signal are 
time-ordered to match the audio track. 

Referring to claims 4, 1 1, 27, and 38, the processing means of Cosatto further comprises 
recognition means responsive to the video input signal, for recognizing at least a selected one 
reference object within at least a selected one video image (the first audiovisual presentation; 
encoded digital signal) of the time-ordered sequence. Col. 7, line 14 - col. 8, line 15 describes 
the recognition system of Cosatto, including how facial features (selected reference objects) are 
located within the video. 

Referring to claims 5, 12, and 28, the recognition means of Cosatto further recognizes at 
least one reference point on the associated reference object. See col. 7, lines 33-39. 

Referring to claims 6, 29, and 39, the recognition means of Cosatto comprises image 
recognition means. See col. 7, line 14 - col, 8, line 15. 

Referring to claims 7-8, the first audiovisual presentation of Cosatto further comprises 
reference object information that identifies and is associated with a selected one reference object. 
See col. 7, lines 40-61, which describe how reference object information (i.e. color and texture 
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analysis, size and position - reference point) is associated with and identifies facial features 
(selected reference objects). 

Referring to claims 13, 30, and 40, Cosatto discloses the processing means further 
comprises first-order object transformation means for transforming at least a selected one 
recognized reference point responsive to the selected one of the plurality of MPEG motion 
vectors. See col. 13, lines 60-65 which discuss morphing (transforming) and col. 14, lines 53-66, 
which discusses how the facial parts are handled by a model for the MPEG4 subsystem. 

Referring to claims 14-15, 31, and 41, the compositing means/reconstructing of Cosatto 
further comprises image transform means (i.e. morphing; col. 13, lines 60-65) for transforming 
the selected one replacement object image (facial feature) associated with the selected one 
reference object. 

Referring to claims 17-18 and 32, the processing means of Cosatto further comprises 
recognition means for identifying a selected one reference object within the first audiovisual 
presentation (col. 7, lines 40-61) and the compositing means further comprises geometric 
transformation means for geometrically transforming the user object geometric information 
responsive to the recognition means. See col. 10, lines 37-53 and col. 13, lines 22-34. 

The compositing means of Cosatto and Rom, supra, further comprises geometric 
transformation means for geometrically transforming the user object geometric information 
producing transformed user object geometric information, responsive to the selected one 
reference object. See col. 6, lines 7-20, where Cosatto acknowledges the ability the change the 
shape and orientation of objects. Also, see Rom at col. col. 1, lines 45-63. 
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Referring to claims 19-20, 33, and 43, the replacement object image of Cosatto comprises 
texture map image data representative of at least one selected user object as viewed from at least 
one predefined viewing position relative to the selected user object and the compositing means 
selects a portion of the texture map image data responsive to the processing means. See col. 6, 
lines 7-12. Also, see col. 1, lines 62-64. 

Referring to claims 21-22 and 34, the compositing means of Cosatto and Rom, supra, 
further comprises geometric transformation means for geometrically transforming the user object 
geometric information producing transformed user object geometric information, responsive to 
the selected one reference object. See col. 6, lines 7-20, where Cosatto acknowledges the ability 
the change the shape and orientation of objects. Also, see Rom at col. col. 1, lines 45-63. 

The geometric transformation means of Cosatto further comprises mapping means for 
mapping the selected portion of the texture map image data onto the transformed user object 
geometric information producing a replacement object image. See col. 6, lines 7-20. 

Referring to claims 23, 35, and 45, the user audiovisual information of Cosatto further 
comprises a time-ordered sequence of user images and the association means further selectively 
associates a plurality of the time-ordered sequence of user images each associated respectively 
with selected ones of the time-ordered sequence of video images (prior to the replacing). See 
col. 3, lines 44-60, which describe how the user images are selected to coincide with an audio 
track in a time-ordered sequence to provide realistic lip and head movements. 

Referring to claims 24 and 36, the audiovisual presentation of Cosatto further comprises a 
time-ordered sequence of video masks, and the processing means further comprises selection 
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means for selecting a portion of the first audiovisual presentation responsive to at least a selected 
one of the video masks. See col. 15, lines 1-9. 

Referring to claim 25, the selection means of Cosatto is further responsive to at least a 
selected one of the MPEG motion vectors. See col. 14, line 55. 

Referring to claim 42, Cosatto and Rom, supra, discloses the replacement object further 
comprises user object geometric information that is transformed, producing transformed user 
object information, responsive to the recognition of at least one selected reference object within 
the first audiovisual presentation, prior to the replacing. See Cosatto at col. 6, lines 7-20, where 
Cosatto acknowledges the ability the change the shape and orientation of objects. Also, see col. 
6, lines 38-45. Also, see Rom at col. 1, lines 45-63. 

Referring to claim 44, the selected portion of the texture map of Cosatto is mapped onto 
transformed user object geometric information, prior to the replacing. See col. 6, lines 7-20. 

Referring to claim 56, Cosatto discloses the audiovisual presentation is comprised of a 
plurality of frames (col. 3, lines 38-39, describe how the audiovisual presentation is video, which 
inherently includes a plurality of frames; see also, col. 7, line 1), at least two of the frames 
having a select portion, the system further comprising analysis means for selectively tracking the 
respective selected portions (i.e. facial feature) of a plurality of frames of the audiovisual 
presentation and identifying and selecting corresponding respective ones of the user images. 

The image integration means is responsive to the analysis means for selectively 
substituting into the audiovisual presentation the corresponding respective ones of the user image 
for the respective selected portions of the plurality of frames (i.e. substituting facial features 
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throughout the animation). See col. 3, line 25 - col. 4, line 26 and col. 12, lines 35 - col. 13, line 
35. 

Referring to claim 58, Cosatto discloses tracking the respective selected portions (facial 
features) comprising at least two frames (col. 3, lines 38-39, describe how the audiovisual 
presentation is video, which inherently includes a plurality of frames; see also, col. 7, line 1), 
identifying and selecting corresponding respective ones of the user images, and providing for 
each frame a replacement user image for integration into the audiovisual presentation responsive 
to the texture map and the user object geometry, and substituting the corresponding respective 
ones of the user image for the respective selected portions of the plurality of frames into the 
audiovisual presentation. See col. 3, line 25 - col. 4, line 26 and col. 12, lines 35 - col. 13, line 
35. 

Although Cosatto discloses replacing a reference object with the best fit sample of user 
object information (i.e. col. 4, lines 15-22) and smoothing and blending the samples which hints 
to transforming the geometry of the user object information for a smooth transition into the first 
audiovisual presentation (col. 3, lines 34-53), Cosatto does not explicitly disclose geometrically 
transforming the user object geometry information to provide transformed user object 
information responsive to the defined orientation. 

However, Rom discloses a method of replacing reference objects (i.e. images of clothing 
originally on the body) in a presentation with object information (i.e. images of clothing to be 
"tried on"), and is thus similar to Cosatto's image replacement and integration method. See Rom 
at col. 1, lines 45-63. Rom discloses using critical points similar to the reference points of 
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Cosatto, wherein the critical points of Rom are used to adjust the spatial configuration of the 
clothing (object information) to fit the body of each user (orientation). The spatial configuration 
of Rom determines the geometry of the clothing (object information); therefore, adjusting the 
spatial configuration is a geometrical transform of object geometry information to fit the 
geometry and orientation of the reference object (i.e. body), wherein the object information is 
placed on the reference object and integrated (forms a composite) into the original audiovisual 
presentation. See Rom at col. 2, lines 20-33 and col. 4, lines 1-11. 

It would have been obvious to one of ordinary skill in the art with the teachings of 
Cosatto and Rom before him at the time of the invention to modify the image substitution and 
integration method of Cosatto to include geometrically transforming (i.e. adjusting spatial 
configuration of) the user object geometry information responsive to the defined orientation, as 
taught by Rom, in order to achieve a more accurate, realistic model in the animation, wherein the 
user information smoothly blends into the original presentation (i.e. base model). 



Response to Arguments 
10. Applicant's arguments with respect to claims 1, 26, and 37 have been considered but are 
moot in view of the new ground(s) of rejection. 

Applicant argues the newly presented limitation of geometrically transforming the user 
image geometry responsive to the orientation of the selected reference object. This limitation is 
taught by Rom, which transforms an image geometry (i.e. image geometry of clothing to try on) 
responsive to the orientation of a selected reference object (i.e. image of body that clothes are to 



Application/Control Number: 09/723, 1 69 Page 1 3 

Art Unit: 2173 

go on), as described above; and the combination of Cosatto and Rom teaches each limitation in 
the claims. 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

12. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. §1.11 1(c) to consider 
these references fully when responding to this action. The documents cited therein teach 
methods of substituting original images with new images. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shawn M. Becker whose telephone number is 703-305-7756. 
The examiner can normally be reached on M-Th 8:00 - 5:30 and alternating Fridays. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W. Cabeca can be reached on 703-305-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is 703-746-7239. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-3900. / / 



